08. Modifying Backyard Flyer

Updating Your Project Code

In this section we will update the project code to be able to control a real drone.

While there is really only one line in the code that needs to change for this to work (the connection to PX4), we will be making several modifications to the project code for safety.

We will be making 3 different changes to our backyard flyer script in order to run properly and safely on our drone:

  • updating the connection type - here we will change the connection being used to connect to a real drone instead of the simulator.
  • modifying the state machine - for safety, we will slightly modify the state machine to give the pilot more control over the starting and stopping of the script.
  • shifting the coordinates of the box - with a real drone we can't reset the location of (0,0,0) of our world, so we will shift our box coordinates to be centered on the local coordinates of the takeoff position.

Updating Connection

In our backyard flyer script, the connection being passed to Drone is MavlinkConnection('tcp:{0}:{1}'.format(args.host, args.port), threaded=False, PX4=False), which is a TCP connection to the simulator passing data as Mavlink messages.

When connecting to the Intel Aero, we will need to change the parameters passed into the connection class to set up a connection to the drone. For the Intel Aero, we will need a UDP connection ('udp:ip_address:port) and tell the MavlinkConnection class we are connecting to a PX4 autopilot, by setting PX4=True. The modified connection class will be:

conn = MavlinkConnection('udp:192.168.1.2:14550', PX4=True, threaded=False)

For the Intel Aero, the IP address specified is the IP address of our computer when connected to the network created by the Intel Aero. To determine our IP address, we can:

  • Windows: open the command prompt and type ipconfig. Look for the IP address for the wireless adapter.
  • OSX and Linux: open the terminal and type ifconfig. Look for the IP address for the wireless adapter.

For Generic PX4

If we are setting up a generic drone running the PX4 autopilot, our connection to the drone is most likely over a dedicated telemetry link connected as a serial device on our computer. If that is the case, instead of 'udp:ip_address:port', we can pass in 'serial_port,baud_rate'.

Documentation for More Details

For more details into the workings of the connection class and different parameters, check out the Udacidrone documentation on the connection class.

Modifying the State Machine

Currently our project code handles all the commands from taking control of the drone, to arming, through the flight and disarming. To give us, the pilot, more control over the environment and the running of the script, we will modify our project code to no longer handle the arming or disarming transition. Instead, that will be up to the pilot to do manually. This is done so the pilot can ensure everything is properly set up and that the flight environment is safe before letting our script control the drone.

Also note that at all times during the flight we have the ability to take back control of the autopilot by flipping the switch on the remote control back to manual control from offboard! This is very important and the pilot should be ready to do this at all points during the flight just in case something unexpected happens!

First, we will remove the handling of the arming transition, by modifying the checks being made when in the MANUAL state. We will change the code to passively wait for the drone's state to be updated by the pilot in order to advance our current flight state.

The current solution state_callback is:

def state_callback(self):
    if self.in_mission:
        if self.flight_state == States.MANUAL:
            self.arming_transition()
        elif self.flight_state == States.ARMING:
            if self.armed:
                self.takeoff_transition()
        elif self.flight_state == States.DISARMING:
            if not self.armed and not self.guided:
                self.manual_transition()

And once we make the necessarily changes, it becomes:

def state_callback(self):
    if self.in_mission:
        if self.flight_state == States.MANUAL:
            # now just passively waiting for the pilot to change these attributes
            # once the pilot changes, need to update our internal state
            if self.guided:
                self.flight_state = States.ARMING
        elif self.flight_state == States.ARMING:
            if self.armed:
                self.takeoff_transition()

        elif self.flight_state == States.DISARMING:
            if not self.armed and not self.guided:
                self.manual_transition()

Next we will need to remove the disarming transition and manual transitions made by the script. Once again, this is so that the pilot is the one who knowingly takes back control of the drone and disarms the motors.

For this change, we will modify both the velocity_callback and the state_callback. We will be completely removing all the code from the velocity_callback as we are no longer interested in triggering the disarming transition. In your backyard flyer solution, we handled stopping the mission during the disarming transition, here, we will add some checks to stop the mission once the pilot has taken back control. In order to achieve this, we will add checks on the drone's state when in the LANDING flight state.

Our modified set of callbacks will now look like:

def velocity_callback(self):
        pass

def state_callback(self):
    if self.in_mission:
        if self.flight_state == States.MANUAL:
            # now just passively waiting for the pilot to change these attributes
            # once the pilot changes, need to update our internal state
            if self.guided:
                self.flight_state = States.ARMING
        elif self.flight_state == States.ARMING:
            if self.armed:
                self.takeoff_transition()
        elif self.flight_state == States.LANDING:
            # check if the pilot has changed the armed and control modes
            # if so (and the script no longer in control) stop the mission
            if not self.armed and not self.guided:
                self.stop()
                self.in_mission = False
        elif self.flight_state == States.DISARMING:
            # no longer want the vehicle to handle the disarming and releasing control
            # that will be done by the pilot
            pass

Shifting the Coordinates

The last change we will need to make to your Backyard Flyer script is handling the fact that we cannot set the local position (0,0,0) coordinate to wherever we would like. Therefore, when we create our set of coordinates to fly our box, we will shift the box to have the first corner be at the drone's current local position. We achieve this by modifying our calculate_box function from:

def calculate_box(self):
    print("Setting Home")
    local_waypoints = [[10.0, 0.0, 3.0], [10.0, 10.0, 3.0], [0.0, 10.0, 3.0], [0.0, 0.0, 3.0]]
    return local_waypoints

to:

def calculate_box(self):
    print("Setting Home")
    cp = np.array([self.local_position[0], self.local_position[1], -self.local_position[2]])  # get the current local position -> note we need to change the sign of the down coordinate to be altitude
    local_waypoints = [cp + [10.0, 0.0, 3.0], cp + [10.0, 10.0, 3.0], cp + [0.0, 10.0, 3.0], cp + [0.0, 0.0, 3.0]]
    return local_waypoints

Almost There!

Now that we have our backyard flyer solution properly modified for running on the Aero, let's move on to test running the script and flying it for real!

Once again keep in mind that most of these drones, including the Intel Aero, do NOT come with obstacle avoidance! The only obstacle avoidance is the pilot, therefore it is very important that the pilot always keep an eye on the drone at all times, even during autonomous flight.

Disclaimer

PLEASE REMEMBER THAT YOUR RIGHT TO USE ANY CODE PROVIDED BY UDACITY IS SUBJECT TO OUR TERMS OF USE, THE FLYING CAR SPECIAL TERMS, HONOR CODE, AND COMMUNITY CODE OF CONDUCT. TO THE EXTENT UDACITY PROVIDES YOU CODE TO USE IN CONNECTION WITH DEMONSTRATING FLYING DRONES AND/OR OTHER AUTONOMOUS FLYING VEHICLES, SUCH CODE IS PROVIDED “AS IS” WITH NO REPRESENTATIONS OR WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. YOU ASSUME TOTAL RESPONSIBILITY AND THE ENTIRE RISK FOR YOUR USE OF ANY CODE AND/OR OPERATION OF ANY AUTONOMOUS FLYING/UNMANNED VEHICLES. TO THE FULLEST EXTENT ALLOWED UNDER THE LAW, UDACITY DISCLAIMS ANY AND ALL LIABILITY RELATING TO YOUR USE OF ANY CODE OR YOUR OPERATION OF ANY UNMANNED AIRCRAFT OR FLYING VEHICLE AS SET FORTH IN THE TERMS OF USE, INCLUDING, BUT NOT LIMITED TO PROPERTY DAMAGE, PERSONAL INJURY, OR DEATH. TO THE EXTENT THAT YOU USE UDACITY’S CODE AND/OR YOUR OWN CODE FOR THE PURPOSES OF THIS COURSE, INCLUDING, BUT NOT LIMITED TO, USE CONNECTED WITH A FLYING DRONE, OR AS OTHERWISE ALLOWED BY UDACITY, SUCH USE BY YOU SHALL COMPLY WITH ALL FEDERAL, STATE, PROVINCIAL, AND/OR LOCAL LAWS REGARDING THE LICENSURE, REGISTRATION, AND OPERATION OF UNMANNED AIRCRAFT OR OTHER VEHICLES. FOR EXAMPLE, BUT NOT IN ANY WAY AS A LIMITATION, THE U.S. FEDERAL AVIATION ADMINISTRATION HAS SPECIFIC RULES RELATING TO THE OPERATION OF UNMANNED AIRCRAFT. THERE ARE ALSO PRIVACY REGULATIONS THAT MAY BE APPLICABLE TO YOUR USE OF THE CODE AND/OR OPERATION OF UNMANNED AIRCRAFT AND/OR VEHICLES AND ARE REQUIRED TO COMPLY WITH THOSE LAWS.

Next Concept